Chapter 12 TRAINING RECURRENT NETWORKS FOR FILTERING AND CONTROL

Neural networks can be classified into recurrent and nonrecurrent categories. Nonrecurrent (feedforward) networks have no feedback elements; the output is calculated directly from the input through feedforward connections. In recurrent networks the output depends not only on the current input to the network, but also on the current or previous outputs or states of the network. For this reason, recurrent networks are more powerful than nonrecurrent networks and have important uses in control and signal processing applications. This chapter introduces the Layered Digital Recurrent Network (LDRN), develops a general training algorithm for this network, and demonstrates the application of the LDRN to problems in controls and signal processing. In Section II we present the notation necessary to represent the LDRN. Section III contains a discussion of the dynamic backpropagation algorithms that are required to compute training gradients for recurrent networks. The concepts underlying the backpropagation-through-time and forward perturbation algorithms are presented in a unified framework and are demonstrated for a simple, single-loop recurrent network. In Section IV we describe a general forward perturbation algorithm for computing training gradients for the LDRN. Two application sections follow the discussion of dynamic backpropagation: neurocontrol and nonlinear filtering. These sections demonstrate the implementation of the general dynamic backpropagation algorithm. The control section (Section V) applies a neurocontrol architecture to the automatic equalization of an acoustic transmitter. The nonlinear filtering section (Section VI) demonstrates the application of a recurrent filtering network to a noise-cancellation application.